Generative Restricted Kernel Machines: A framework for multi-view generation and disentangled feature learning

Abstract This paper introduces a novel framework for generative models based on Restricted Kernel Machines (RKMs) with joint multi-view generation and uncorrelated feature learning, called Gen-RKM. To enable generation, this mechanism uses shared representation of data from various views. Furthermore, the model has primal dual formulation to incorporate both kernel-based (deep convolutional) neural network within same setting. When using networks as explicit feature-maps, training procedure is proposed, which jointly learns features subspace representation. The latent variables are given by eigen-decomposition kernel matrix, where mutual orthogonality eigenvectors represents learned features. Experiments demonstrate potential through qualitative quantitative evaluation generated samples standard datasets.